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11 dune 1976 


MEMORANDUM FOR: Mr. Richard Lehman 
Deputy to the DCI for National Intelligence 


SUBJECT : Technical Review and the Functions of the 
TAP 


XL thought you might be interested in some thoughts 1 
have on how to, and how not to, use the IAP, based on experi- 
ence with a comparable formal technical review process at 
Rand. The Rand experience is, of course, not fully applica- 
ble to our situation, but, upon reflection, it seems to adapt 

tself rather well to our needs. 


1. When. Should the Review Take Place? For our 
purposes, technical review should be regarded as the import- 
ant penultimate step in the preparation of a National Estimate, 
SNIE, IM, etc., before it is submitted for formal coordination. 
It is, of course, quite different from the kinds of informal 
comments which drafters routinely obtain from their colleagues 
or from other offices within the Community at earlier Stages 
in the production process. In other words, when a product is 
submitted for technical review, it is assumed to be essentially 
complete and ready for issuance, except for final coordination 
and Community acceptance. 


2. Role of Panelists. In undertaking their review, 
panel members act as individuals. They are invited into the 
process because of their personal scholarship, experience and 
recognized expertise in their professions. I think it would 
be a serious mistake to constitute them in any sense as a 
"sSub-panel." This would inevitably create an artificial 
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assignment with the IAP. aA panel member undertaking a particu- 
lar review should look on himself much as if he were acting as 
a "reader" on behalf of a journal editor or book publisher. 
As the panel's name indicates, their role is purely advisory. 
they are asked to make a reasoned personal judgement about 
the technical soundness, analytic quality and, in some instances, 
policy relevance of the product for the benefit of those 
responsible for managing the national production process. 


3. Content of Reviewer's Critique. Each reviewer 
should be asked to.assess, on the basis of a careful reading 
of the draft, the technical quality of the work, including 
assumptions, matters of Lact, logic of development and ade~ 
quacy of the evidentiary base to Support the conclusions. 
The reviewer should satisfy himself that the drafters have 
accomplished their purpose in a workmanlike manner, consistent 
with high professional standards. In addition the reviewer 
might be encouraged to include comments on completeness, rele-— 
vance and other characteristics affecting the quality of the 
analysis, as well as to indicate where his own views might 
differ from those presented in the draft. The reviewer’s 
critique should be submitted ain writing addressed to the DCI but, 
for practical purposes, passed to ‘the NIO responsible for the 
product. It should be framed not as a book review, but rather 
aS a set of specific recommendations that might be carried into 
effect by the drafters. Minor comments may be.included in the 
manuscript itself as marginalia. 


_ 4. Resolution of-Conflicts among Reviewers or 
between Reviewers and Producers. Responsibility for acceptance 
Or rejection of the recommendations of the individual panelists 
should rest in the first instance with the drafters, in consul— 
‘tation with their line managements. The responsible NIO would 
be the court of last resort. It would be highly desirable to 
include the requirement that the drafters prepare a memo to the 
NIO responsible, outlining the changes that have been made as 
a result of the critiques, and the rationale for rejecting 
particular critique recommendations. Though time consuming, 
this would be a useful intellectual discipline and woula pre- 
serve a valuable record for future purposes. There should be 
no requirement. to submit such a memo to the reviewer for his 
approval; his responsibility ceases with the submission of his 
written critique. Whether to pass such a memo to the reviewer 
as a courtesy in particular instances, however, should be left 
to the discretion of the responsible NIO. 


S- Flexibility. In discussing this subject with 
some of my NIO colleagues, I have been struck with the variety 
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that will be represented within the IAP. The above sugges- 
tions, therefore, should be viewed simply as an outline of a 
typical pattern, with wide latitude being preserved for de- 
parting from this pattern. My injunction against constituting 
the individual panelists as sub-panels, for example, could 
well be waived in particular instances, where an NIO might 
consider it desirable ta convoke a group of panelists, seminar-— 
fashion, to discuss a key estimate in a more collegial and 
informal way, say; at an earlier stage in the process. Other 
innovative ways of using panelists individually or in groups 
should. surely be encouraged. For this reason I feel that we 
would be better of £f not to overdetermine the system in ad- 
vance. This is an added reason why it would be clearly inad~ 
sable to reguire routinely the organization of panel members 


eee sub-panel groups. 


NiO/Economics 
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